minimax entropy
Meta-TTT: A Meta-learning Minimax Framework For Test-Time Training
Tao, Chen, Shen, Li, Mondal, Soumik
Test-time domain adaptation is a challenging task that aims to adapt a pre-trained model to limited, unlabeled target data during inference. Current methods that rely on self-supervision and entropy minimization underperform when the self-supervised learning (SSL) task does not align well with the primary objective. Additionally, minimizing entropy can lead to suboptimal solutions when there is limited diversity within minibatches. This paper introduces a meta-learning minimax framework for test-time training on batch normalization (BN) layers, ensuring that the SSL task aligns with the primary task while addressing minibatch overfitting. We adopt a mixed-BN approach that interpolates current test batch statistics with the statistics from source domains and propose a stochastic domain synthesizing method to improve model generalization and robustness to domain shifts. Extensive experiments demonstrate that our method surpasses state-of-the-art techniques across various domain adaptation and generalization benchmarks, significantly enhancing the pre-trained model's robustness on unseen domains.
Domain Adaptation via Minimax Entropy for Real/Bogus Classification of Astronomical Alerts
Cabrera-Vives, Guillermo, Bolivar, César, Förster, Francisco, Arancibia, Alejandra M. Muñoz, Pérez-Carrasco, Manuel, Reyes, Esteban
An important number of these alerts analysis of multiple massive datasets in real time, are bogus artifacts created by the image reduction pipelines, prompting the development of multi-stream machine hence, the importance of creating real/bogus classification learning models. In this work, we study algorithms which have proven to be extremely useful for Domain Adaptation (DA) for real/bogus classification detecting real astrophysical phenomena. During the last of astronomical alerts using four different decade, most of these algorithms have been based on Convolutional datasets: HiTS, DES, ATLAS, and ZTF. We Neural Networks (Cabrera-Vives et al., 2016; 2017; study the domain shift between these datasets, Reyes et al., 2018; Duev et al., 2019; Turpin et al., 2020; Yin and improve a naive deep learning classification et al., 2021; Rabeendran & Denneau, 2021) which need a model by using a fine tuning approach and significant amount of data to be trained.
Learning from the Wisdom of Crowds by Minimax Entropy
An important way to make large training sets is to gather noisy labels from crowds of nonexperts. We propose a minimax entropy principle to improve the quality of these labels. Our method assumes that labels are generated by a probability distribution over workers, items, and labels. We infer the ground truth by minimizing the entropy of this distribution, which we show minimizes the Kullback-Leibler (KL) divergence between the probability distribution and the unknown truth. We show that a simple coordinate descent scheme can optimize minimax entropy.
Learning from the Wisdom of Crowds by Minimax Entropy
Zhou, Dengyong, Basu, Sumit, Mao, Yi, Platt, John C.
An important way to make large training sets is to gather noisy labels from crowds of nonexperts. We propose a minimax entropy principle to improve the quality of these labels. Our method assumes that labels are generated by a probability distribution over workers, items, and labels. We infer the ground truth by minimizing the entropy of this distribution, which we show minimizes the Kullback-Leibler (KL) divergence between the probability distribution and the unknown truth. We show that a simple coordinate descent scheme can optimize minimax entropy.